Emotional Coloring of Computer-Controlled Music Performances

Computer Music Journal Computer Music Journal, 24:4, pp. 44–63, Winter 2000 © 2000 Massachusetts Institute of Technology. When interpreting a musical score, performers introduce deviations in time, sound level, and timbre from the values indicated by composers. These deviations depend on the expressive mechanical and acoustical possibilities offered by the instrument they are playing and can vary in nature. Musical structure, biological motion, timekeeper variance and motor variance, and the performer’s expressive intentions are the most common sources of deviation in a performance. In this paper, we focus on the deviations that render different emotional characteristics in music performance as part of expressive intentions. Many important contributions to research in this area derive from Alf Gabrielsson’s group at the University of Uppsala. Much of their work has centered on the so-called basic emotions (anger, sadness, happiness, and fear, sometimes complemented with solemnity and tenderness). The group found that all of these emotions, as conveyed by players, could be clearly recognized by an audience containing both musically trained and untrained listeners (Juslin 1997a; Juslin 1997b). Other researchers have shown that more complex emotional states can be successfully communicated in performance, although it is not completely clear how these states are defined—because performers and listeners often use different terms in describing intentions and perceived emotions (Canazza et al. 1997; Battel and Fimbianti 1998; De Poli, Rodà, and Vidolin 1998; Orio and Canazza 1998). The identification of a communication code in real performances is another essential task in research on emotional aspects of music performance, and researchers have approached this problem in several ways. Gabrielsson and Juslin (1996) asked professional musicians to play a set of simple melodies with different prescribed emotions. From these renditions, the authors identified sets of potential acoustical cues that players of different instruments (violin, flute, and guitar) used to encode emotions in their performances which were similar to the sets listeners used for decoding these emotions (Juslin 1997c). A difficulty with this experimental design is that conflicts may exist between the general character of the composition and the prescribed emotional quality of its performance. Another important aspect is to what extent the characteristic performance variations contain information that is essential to the identification of emotion. The significance of various performance cues in the identification of emotional qualities of a performance has been tested in synthesis experiments. Automatic performances were obtained by setting certain expressive cues to greater or lesser values, either in custom developed software (Canazza et al. 1998) or on a commercial sequencer (Juslin 1997c), and in formal listening tests listeners were able to recognize and identify the intended emotions. In the computer program developed by Canazza et al., the expressiveness cues were applied to a ”neutral” performance as played by a live musician with no intended emotion, as opposed to a computer-generated ”deadpan” performance. Juslin manually adjusted the values of some previously identified cues by means of ”appropriate settings on a Roland JX1 synthesizer that was MIDI-controlled” by a Synclavier III. No phrase markings were used, and Juslin assumes that this reduced listeners’ ratings of overall expressiveness. The present work attempts to further explore the flexibility of Director Musices (DM), a program for automatic music performance that has been developed in our group (Friberg et al. 2000). More specifically, we have explored the possibilities of using the DM program to produce performances that differ with respect to emotional expression.